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(57) Abstract: A method for identifying and modeling 
nonlineari ties in communications channels, particularly 
optical communication channels. A channel in general is 
modeled as a summation of linear and non linear terms 
having memory. The terms are functions of the input to 
the channel with respect to time, such as a sequence of 
input bits to the channel. In one embodiment the most 
recent 5 input bits to the optical channel are used to ac- 
cess a value in a look up tale. The value accessed is then 
compared to an actual value received from the channel. 
The difference between the value in the table and the ac- 
tual channel value may be used to correct the value in the 
table, for example using a LMS (Least Means Squared) 
algorithm. When the look up table and the channel con- 
verge the look up table contains a model of the channel 
with memory that can model nonlinearities. A nonlinear 
channel having memory may also be modeled in terms 
of Volterra Kernels, which may equivalently change into 
a look up table odel using the Hadamard transform. The 
Volterra Kernel representation also has the added advan- 
tage that it can represent a look up table of N table entries 
in at most N-l Volterra kernels. In many cases only a 
few Volterra Kernels are required to model the behavior 
of the channel. For example embodiments are disclosed 
in. which two Kernels may model a nonlinear fiber op- 
tic channel, operating at a wavelength of 850 nanometer 
through a multimode channel at a data rate of I Gigabit 
per second. 
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METHOD AND SYSTEM TO IDENTIFY AND CHARACTERIZE NONLINEARITIES 
IN OPTICAL COMMUNICATIONS CHANNELS 

FIELD OF THE INVENTION 

The invention relates to methods used to characterize communication channels 
and, in particular embodiments, to methods used to characterize and model nonlinear 
optical communication channels. 

BACKGROUND OF THE INVENTION 

Signal distortion in an optical channel may be caused by nonlinearities that exist 
in various components, which comprise an optical channel. Such distortion can be 
characterized by mathematical models of the nonlinear behavior of the optical 
communication channel. By using such models, compensatory measures, for example 
equalizers that compensate for distortion, which degrades the performance of the 
channel, may be designed. 

One source of distortion is dispersion. Dispersion may be a combination of linear 
and nonlinear components. 

Dispersion found in optical fibers, particularly multimode fibers, may limit the 
bandwidth in optical channels. The problem of dispersion may be more acute in 
multimode fibers in general, but it may also be a problem in single mode fibers, 
especially in long runs of fiber. Single mode fibers commonly have lower dispersion 
than multimode fiber. The dispersion commonly found in single mode fibers may be 
chromatic dispersion or polarization mode dispersion. The dispersion that is seen in 
single mode fibers is generally a lot less per length than the dispersion found in multi- 
mode fibers. Dispersion is cumulative, however, and so tends to increase as fibers 
increase in length. Accordingly, methods discussed herein in terms of multimode fibers 
may apply equally well to single mode fibers. 

In multi-mode fibers, the multiple modes of propagation within the fiber commonly 
cause dispersion. Generally, in the various modes of propagation within a fiber, the 
light signal has different speeds of propagation. So if a light pulse is sent from a 
transmitter, the pulse will propagate in multiple modes traveling at different speeds. 
The pulse, traveling in each mode, will reach the receiver at a different time. The 
multiple times of arrival of the input pulse traveling in several modes create a distorted 
pulse, somewhat like a spread out version of the transmitted pulse. 

Optical channels are inherently nonlinear. Electromagnetic fields can be 
described by Maxwell's equations, which are linear. Because Maxwell's equations are 
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1 linear, the principle of superposition applies to the electromagnetic field. What is 
modulated, however, in optical channels is not the electromagnetic field perse, it is the 
optical power. Superposition doesn't exactly apply to optical power. 

An optical photo detector is commonly a square law device. The photodetector 

5 responds linearly to the optical power, in fact the photo current generated by a 
photodetector is commonly a very accurately linear function of the optical power, butthe 
power is a quadratic function of the electromagnetic field, thus creating a source of 
nonlinearity. 

It has been shown that if a laser has a large spectral width, instead of being a 
1 0 very monochromatic laser, the significant spectral width tends to result in a linearization 
of the channel even considering the effect of the optical power and the square law 
nature of the photodetector. In other words if a laser has a very monochromatic, narrow 
spectrum then the channel tends to behave less linearly than does a laser that has a 
wider spectrum. 

15 Nonlinear behavior in an optical channel may depend to varying extent on the 

properties of the photodetector and the fiber. Additionally there is a possibility of 
distortion from the laser itself. If there is dispersion in the laser, for example because 
the laser has some bandwidth limitation itself, then nonlinearity of the laser could also 
contribute to the total nonlinearity in the optical channel. 

20 When the intensity of an optical signal is high, a fiber optic which transmits the 

optical signal itself may introduce nonlinearities. These nonlinearities may result from 
the fact that the index of refraction of the fiber optic depends, to some extent, on the 
intensity of the optical signal itself. This effect commonly gives rise to nonlinearities 
known as "four-wave mixing", "self-phase modulation", and "cross-phase modulation". 

25 Additional nonlinearities may result from the phenomena known as stimulated Raman 
scattering and stimulated Brillouin scattering. For a more comprehensive treatment of 
nonlinearities, see the text "Fiber-Optic Communication Systems", second edition, by 
Govind P. Agrawai, John Wiley and Sons, 1997, ISBN 0-471-17540-4, which is 
incorporated by reference herein. 

30 Although the above sources of nonlinearity in an optical channel are presented 

as examples, other sources may exist. The techniques described in this disclosure may 
be applied to all kinds of nonlinearities regardless of their physical source, as will be 
clear to those skilled in the art. 

35 SUMMARY OF EMBODIMENTS OF THE INVENTION 

In one aspect of the invention, the disclosure illustrates a method for modeling 
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the behavior of a data channel. To create the model, the sequence of data input to the 
data channel is examined. A portion of the sequence of data input to the data channel 
is used as an index to a channel model value. Output data from the channel is 
sampled. A numerical channel model value is compared with the sampled value and 
the channel model value is adjusted based on the results of the comparison. 

. In another aspect of the invention, a method for modeling the behavior of a data 
channel using Volterra Kernels is illustrated. A sequence of data is provided to the data 
channel. A part of the sequence of data input to the channel is provided to the Volterra 
Kernel model of the channel. The Volterra Kernel (VK) model produces a channel 
model value. The data values measured from the channel are compared to the VK 
channel model value and the results used to update the Volterra Kernel model. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The features, aspects, and advantages of the present invention, which have been 
described in the above summary, will be better understood with regard to the following 
description, appended claims, and accompanying drawings where: 

Figure 1 is a graphical illustration of an environment in which embodiments of the 
present invention may be used. 

Figure 2 is a graphical illustration of an equipment setup, which may be used to 
characterize optical channels. 

Figure 3 is a table listing of fibers, which were measured using the measurement 
setup illustrated in figure 2. 

Figure 4 is a graphic illustration of an example of a system that may be used for 
channel identification. 

Figure 5 is a graphical illustration of channel identification based on Volterra 
Kernels (VK). 

Figure 6 is a graph illustrating measurement of a channel having an 850 
nanometer VCSEL (Vertical Cavity Surface Emitting Laser) and fiber F0, and 
comparing the signal measurements to linear and nonlinear channel models. 

Figure 7 is a graph illustrating the measurement of a channel having an 850 
nanometer VCSEL and fiber F0, and comparing the amount of error generated by 
linear and nonlinear channel models. 

Figure 8 is a graph representing an impulse response and a second order 
Volterra Kernel for an 850 nanometer VCSEL and fiber F0. 

Figure 9 is a table of the signal to noise ratios computed for linear and nonlinear 
channel models for a channel operated at a wavelength of 850 nanometers. 
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1 Figure 10 is a graph comparing linear model, measured signal and error within 

a channel using a fiber F0 at a wavelength of 1310 nanometers. 

Figure 1 1 is a graph of the impulse response and a second order Volterra Kernel 
for a channel using fiber F0 and a distributed feedback (DFB) laser with 1310 
5 nanometer wavelength. 

Figure 12 is a table of signal-to-noise ratios for a linear model of a channel 
employingl 310 nanometer DFB laser. 

Figure 13 is a block diagram of a nonlinear optical channel equalizer, according 
to an embodiment of the invention. 
10 Figure 14A is a block diagram of a nonlinear optical channel equalizer that 

employs a Volterra Kernel estimator according to an embodiment of the invention. 

Figure 14B is a block diagram of a non-linear optical channel equalizer employing 
a lookup table channel estimator, according to an embodiment of the invention. 

Figure 1 5 is a Viterbi decoder employing a nonlinear channel estimator, according 
15 to an embodiment of the invention. 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 

The listed figures describe and illustrate various aspects of the present invention. 
Accordingly, Figure 1 is a graphic illustration of an environment in which 
20 embodiments of the present invention may be used. The environment illustrated at 1 01 

is an information distribution system, such as may be found, for example, in a computer 

network accessing a remote database. 

In Figure 1, users 105 are coupled to a server 103. The users then may 

communicate with remote database 107 using fiber optic 109. Fiber optics may be 
25 used to communicate large amounts of data between points because of their large 

bandwidth. 

The data rate of optical channels is, in part, limited by nonlinearities in the 
channel. The term "channel", as used herein, includes the laser, amplifiers that drive 
the laser, the fiber optic, the photo detector receiver and all elements included in the 
30 transmission and reception of a fiber optic signal. 

For characterizing optical channels, the measurement setup illustrated in Figure 
2 was used. 

Figure 2 is a graphical illustration of an equipment setup used to characterize 
optical channels. A signal is generated by modulating a laser 219. The modulated 
35 laser signal is provided to a fiber optic 215. In the embodiment illustrated in Figure 2, 
the signal provided to the optical channel was a 1 Gigahertz (GHz), 127 bit, pseudo- 
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random binary sequence provided by bit error rate tester 201 . The resulting received 
signal was detected by a photodetector 207, captured using an oscilloscope 205 and 
stored as a file in computer 209. Using computer 209, the received signal may be 
processed and the nonlinearity that exists in the received signal can be identified. The 
approach taken is that no attempt is made to identify the source of the nonlinearity as 
belonging to a specific channel component such as a photodetector or laser or any 
other source. Instead the nonlinearity in the received signal was examined without 
being concerned for the physical source of the nonlinearity . 

Two types of lasers were tested. The first type was a 850 nanometer VCSEL 
(vertical cavity surface emitting laser) and the second type of laser was a 1310 
nanometer DFB (distributed feedback) laser. All measurements were done at a data 
rate of 1 gigabit per second. 

The bit error rate data tester 201 generated a pseudo random binary sequence 
that was periodically repeated. The channel measurements were performed at a data 
rate of 1 gigabit per second, but the method may be applied to any data rate. 

In the setup illustrated in Figure 2 free space optics and an optical bench 221 
were used. The laser was aligned using alignment screws so that the laser 219, the 
fiber 215 and the lens were arranged to focus the light into fiber 215. Laser 219 
requires a DC bias current provided by bias power supply 203. A bias Tee 21 1 allows 
the combination of the data signal with a bias in order to apply a correctly biased data 
signal to the laser 219. 

A length of fiber 215 is contained in a spool so that lengthy fibers up to more than 
1 kilometer may be used. In the case of single-mode fibers, the length of the spool 
could exceed 700 km. Photodetector 209 is a commercial unit having a bandwidth of 
10 gigahertz. The oscilloscope 205 is a real time sampling oscilloscope which samples 
at 8 gigahertz. Because the data rate used was 1 gigabit per second, the received 
signal was sampled on average 8 times per bit. Samples of the received signal were 
captured in the memory of the oscilloscope and then downloaded to the computer 209 
in the form of a file. Typical received signal data files contained about 65,000 data 
points representing received samples of the pseudo random input sequences of 127 
bits. 

Once the data points have been accumulated they may be processed and 
information about the behavior of the optical channel, particularly the nonlinearity, may 
be extracted. Additionally any possible nonstationarity of the channel may be identified. 
In other words, if the channel changes in time, such nonstationarity may be detected. 
In the measurements taken, however, no significant nonstationarity was observed. 
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1 Figure 3 is a table listing of fibers, which were measured using the measurement 

setup illustrated in figure 2. Six different fibers, having core diameters from 50 to 62-1/2 
microns and the lengths from 270 meters to about 2.2 kilometers, were tested. All of 
the fibers exhibit some dispersion phenomenon and bandwidth limitation to varying 

5 extent. 

To model channel nonlinearity two different mathematical methods, representing 
nonlinear functions having memory, were chosen. Because dispersion is inherently 
a function having memory, the resulting channel nonlinearity is a nonlinear function with 
memory, as opposed to memoryless nonlinearity. Memoryless nonlinearity may be 

10 represented by a function of the form Y = F(X), with the nonlinearity an arbitrary 
nonlinear function F of a single variable. Nonlinearity with memory can be described 
as a nonlinear function of a number of consecutive samples of the transmitted data, so 
that the nonlinear function with memory is of the form Y = F( X n ,X n . 1 ,X n ^ 2 ,..). In other 
words nonlinearity with memory is a function of several variables, where the variables 

15 are exemplarily samples of the transmitted bit sequence. Nonlinearity with memory is 
a more general kind of nonlinearity that may be found in optical channels. In other 
words the characteristic of an optical channel is not only a function of the current bit, but 
depends also on previously transmitted bits. 

The nonlinearity in an optical channel can be modeled as a look-up table, for 

20 example as a nonlinear function of the five most recently transmitted bits. In a digital 
transmission system the transmitted bits have 2 levels, 0 and 1 (or +1 and -1 depending 
on the convention that is used). Because of the discrete nature of the transmitted bits, 
the function characterizing a channel depending on the five most recent bits can only 
have 2 5 =32 values. Accordingly there are 32 possible values that model the channel 

25 by listing all the values for all the possible sequences of the most recent 5 input bits. 
Such a model can be implemented as a look-up table. The look up table is one model 
with memory for characterizing an optical channel having nonlinearity. 

A second optical channel model is based on a Volterra series. Volterra series 
have been studied in mathematics for a considerable time. Volterra series are a way 

30 to represent nonlinear functions. In general Volterra series are infinite series, i.e. a 
series with infinite number of terms. In the present case, however, since the transmitted 
data is binary, the number of terms is finite. The number of terms in the series used to 
model a binary optical channel is 2 to the power N, where N is the number of most 
recent transmitted bits, which are taken into account. That is the same number of terms 

35 as the number of entries in a look-up table. Accordingly, no matter what description is 
used, whether the look-up table approach or the Volterra series approach, the number 
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of required elements in each model is 2 to the power N, where N is the number of most 
recent bits transmitted that are taken into account. In the look-up table approach the 
2 to the N parameters are the values of the channel for all the possible values of the 
most recent N input bits. 

For the Volterra series, the parameters are the coefficients of the series and 
because the series has 2 N terms there are 2 N coefficients. 2 N is the same number of 
parameters as required by the look up table model. Accordingly regardless of which 
description is chosen, the maximum number of parameters that is needed to completely 
specify the nonlinear function is the same, i.e. 2 N . It can be shown that the Volterra 
Kernel representation is mathematically equivalent to the look-up table. In fact a 
mathematical transformation known as the Hadamard transformation can transform the 
look up table model to the Volterra Kernel model and Volterra Kernel model to the look 
up table model. 

N is generally an empirically determined value, the exemplary value of N=5 in the 
present embodiment appears to work very well. The value of N, of course could 
depend on the fibers, wavelength of the laser and the rate of the data being transmitted. 
In principle it is possible to find fibers where N is larger or smaller, but jn present 
embodiments N=5 is sufficient. 

An advantage of choosing the Volterra series description is that there is a way to 
make the description somewhat compact by grouping terms. By grouping the terms 2 N 
terms are transformed into 2 (N1) convolution-like terms having a shift register-type 
sequence. That means, for example, the terms that go into the sequence of grouped 
terms are products of bits. So, for example, a X N term would be multiplied by an X N .^ 
term and so on. The shift register sequence of grouped terms means shifted versions 
of the product. So, for example, this shift register sequence would be in the form X N * 
X N -i. x n-i * *n-2. x n-2 * x n-3 and so on. This is shift register-type sequence is one that 
can describe terms of the Volterra series as a convolution of this kind of shift register- 
type sequence with a certain kernel. A kernel is an element that resembles an impulse 
response. A kernel is not an impulse response in the traditional sense, but a traditional 
impulse response is used as a linear portion of the nonlinear kernel function. 

Terms involving a convolution of a shift register type sequence with a certain 
kernel, are referred to as Volterra Kernels. There are multiple Volterra Kernels 
corresponding to different kinds of products of bits. The maximum number of Volterra 
Kernels needed to represent an N bit sequence to is 2 N " 1 . So, for example, in the case 
where N=5, the total number of terms in the Volterra series is 2 N or 32 terms. But these 
32 terms can be mapped into 16 Volterra Kernels. Sixteen Volterra Kernels suffice and 
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1 the number of Volterra Kernels is equal to 2 (N1) , or in the present example 2 4 or 16. So 
there are 16 Volterra Kernels and the Volterra Kernels may provide a more convenient 
and more compact description of a nonlinearity particularly because commonly in many 
applications the number of Volterra Kernels is actually less than 2 (N1) . Although the 

5 number of Volterra Kernels to theoretically define a Volterra Kernel model is 2 (N1) , some 
of the kernels are very small and can be neglected. In some situations the channel may 
be effectively modeled by one or two Volterra Kernels. That is the situation in the 
optical channels of the present embodiment. In present embodiments the nonlinearity 
is simple enough so that it can be described completely by a linear term and a single 

10 Volterra Kernel, indicating that a fairly simple kind of nonlinearity is present. 

Because the nonlinearity is simple doesn't mean that its negligible. In fact in 
present embodiments the nonlinearity, though simple, is a very strong. Because the 
nonlinearity has a simple description, simple equalizers may be built In other words 
nonlinear equalizers with memory which are relatively simple but very powerful may be 

15 built. Such equalizers can be very effective because they can deal with the nonlinearity 
that is strong whereas a traditional kind of linear equalizer might fail completely. 

Figure 4 is a graphic illustration of an example of a system that may be used for 
channel identification. The system is implemented in software. 

Input bits 401 are provided to a channel 403. What is used for the channel 403 

20 is the measurements that were taken with the setup illustrated in Figure 2. So the 
channel input and channel response are actual measured values. 

In the present embodiment the measurements are stored in a file and then 
processed off-line in a computer using a software program that implements the 
remainder of the system illustrated in Figure 4. This was done for the sake of 

25 convenience because the equipment for the setup in Figure 2 was readily available. 
There is however no constraint to prevent the present disclosed methods from being 
implemented as part of a real time or embedded system. 

The look-up table 409 is adaptive. That is the value in it can be adapted using 
the LMS (least-means-squared) algorithm. Basically what is done, is to compare the 

30 measured sequence from the channel 403 (the file of received values stored by 
computer 209) with the output of the look-up table. The addresses in the look up table 
409 are received from a shift register 407 representing the values of the latest 5 input 
bits provided as input to the channel. Initially the look-up table values are initialized to 
all zeros started from situations where there is no knowledge of the nonlinearity. In 

35 such a case the error will be equal to the input signal. So the error will be large initially. 
The values in the look up table can be adjusted using an LMS algorithm to update the 
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look-up table according to the error 405 observed. The LMS algorithm will converge so 
that it tends to drive the error to zero or as close as possible to zero. 

Once the look-up table converges the error will be small or zero. In actual 
practice the error may not be exactly zero because the input signal may include some 
random noise. Additionally it is also possible the look-up table doesn't perfectly model 
all the nonlinearity. There may be some residual unaccounted for nonlinearity because 
N was chosen equal to 5 but in practice a N of 8 or 10 may have been required to more 
closely model the fiber. The choice of N is an implementation detail depending on the 
type of fiber, data rate, and a variety of other factors. 

Additionally in figure 4 the look up table is like having an array where the address 
is created by looking at 5 consecutive bits, but the number is accessed is a single 
number that could be represented by multiple bits which could be 16 bits or 24 bits, and 
as such may contain some quantization error. 

Figure 5 is a graphical example of channel identification based on Volterra 
Kernels. The Volterra Kernel approach is an approach mathematically equivalent to the 
use of a look-up table, as illustrated in Figure 4. In Figure 5, four Volterra Kernels are 
used. Normally there would be 16 Volterra Kernels present in a case where N is equal 
to 5. In some cases some of the Volterra Kernels may be negligible and depending on 
the Bit Error Rate (BER) required only the most important ones may need to be taken 
into account in the model, as for instance illustrated in Figure 5. For example in Figure 
5 there is a linear term which is represented by the Kernel labeled FIR 511. "FIR" 
indicates that each Kernel is represented by a finite impulse response (FIR) filter. In 
Figure 5, FIR 511 directly takes a sequence of input bits and creates a convolution of 
the input sequence with an impulse response of FIR filter 51 1. Hence FIR 51 1 is a 
linear term, i.e. a traditional convolution. So if only a term represented by FIR 511 is 
needed for the channel model, the channel is essentially linear. If, in addition to FIR 
51 1 , other terms are present then the model is nonlinear. 

FIR 51 3 is a Volterra Kernel which is associated with products of two consecutive 
bits, i.e. a second order Volterra Kernel. FIR 513 is a second order Volterra Kernel 
because it accepts a term 505 which is a product of the most recent input bit and the 
bit preceding the most recent input bit. The most recent input bit is designated as A N 
and the bit preceding the most recent bit is designated A N ..,. 

Second order Volterra Kernel FIR 513 is associated with two (consecutive) bits 
i.e. bits that are one time delay apart, it accepts the product of two (consecutive) bits. 
Another second order Volterra Kernel 5 1 5 is produced by taking the product of bits that - 
are two time delays apart, 501 and 503 representing time delays. The most recent bit 
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1 and the bit prior to the second most recent bit are multiplied and result in output 507, 
which is the accepted by FIR 51 5. In other words A N is multiplied by A N _ 2 , here, because 
this is delayed by 2 units. FIR 515 is a Finite Impulse Response filter (just as FIR 51 1 
and FIR 513) but the input, instead of being bits is the product of 2 bits, and so it is a 

5 second order Volterra Kernel. FIR 51 5 accepts the product of bits A N and A N _ 2 , so it is 
a different type of Volterra Kernel than FIR 513 that accepts sequential bits. Both FIR 
515 and FIR 513 are second order. 

A third order Volterra Kernel FIR 51 7 is determined by the product (509) of 3 bits. 
509 is a product of 3 consecutive bits A N , A N-1 and A N _ 2 . Accordingly FIR 517 is a third 

10 order Volterra Kernel. 

In the simplified model illustrated in Figure 5 only the aforementioned 4 kernels, 
the linear kernel, two second orders kernels, and a third order kernel are present. In 
practice, if a nonlinearity were very strong, the Volterra Kernel model could have 16 
kernels or more if N greater than 5 (where N is the number of most recently transmitted 

15 bits to be considered) . In the present embodiment out of the 1 6 Volterra Kernels only 
the four illustrated in Fig. 5 are significant, the others are negligible. 

In the present embodiment FIR 51 1 and FIR 513 are much more significant than 
FIR 515 and FIR 517, and in fact FIR 515 and FIR 517 could be completely neglected. 
The Volterra Kernel model tends to simplify the description of the channel 

20 because it reduces the number of independent terms that need to be taken into 
account. Volterra Kernels may group together all the terms that are time-shifted 
versions of each other. In the more general Volterra Series description, all the terms 
are represented independently. When groups of terms of a Volterra Series are time- 
shifted versions of each other, they may be associated with each other, and all the ones 

25 that are time-shifted versions of each other may be grouped as a single Volterra Kernel. 

The two equivalent representations - the look-up table model and the Volterra 
Kernel model are mathematically equivalent because the Hadamard transformation can 
transform one into the other. The reason to choose one or the other is essentially ease 
of implementation. The model that is simpler is commonly dependent on the particular 

30 problem that is trying to be solved. For example, if a strongly nonlinear function with 
short memory is present (meaning small amounts of dispersion) the look-up table 
probably provides a simpler model. The look up table is likely simpler because, if the 
dispersion is not large N will not be large. If N is not large then 2 N is not a very large 
number of coefficients. But if the memory span of the nonlinearity (i.e. N) grows then 

35 the look-up table grows exponentially because the size of the look-up table is 2 N . The 
size of the look up table may become prohibitively complex for large N. In some cases 
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1 Volterra Kernels may provide a simpler model when N is large and the nonlinearity has 
a simple form which can be described with few kernels, such as the previous example 
that could be limited to two kernels. 

If N is large, it will generally indicate that there is a lot of dispersion in the channel, 

5 but the form of the nonlinear function could be very simple. The function may be simple 
enough that it can be described, as in the previous example, by only two Volterra 
Kernels. If N is large and the Volterra model is simpler, the nonlinearity may be 
completely modeled with two kernels, even though N is very large. Accordingly in some 
cases the Volterra Kernel description is simpler. In other cases the look-up table 

10 description is simpler. 

In the studies performed, the amount of dispersion necessitated an N on the 
order of 5, but in principle, for severely bandwidth limited fibers, N could be larger. In 
the case where two Volterra Kernels are sufficient to create an accurate channel model, 
regardless of how large N is, the Volterra Kernel description is generally simpler. 

15 Figure 6 is a graph illustrating channel measurement, of a channel having an 850 

nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
linear and nonlinear fiber models. In Figure 6 the solid line corresponds to the 
observations, the dotted line represents the linear model, and the dashed and dotted 
represents the nonlinear model. As can be seen from 601 , 603 and 605 the nonlinear 

20 model approximates the measurements of the channel to a greater degree than the 
linear model. Accordingly if the channel model were limited to a linear model a good 
fit for the transmitted data would not be achieved. In contrast using the nonlinear model 
a significantly closer fit may be obtained. 

Figure 7 is a graph illustrating channel measurement of a channel having an 850 

25 nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
amount of error generated by linear and nonlinear channel models. Figure 7 is a graph 
representing the same time as the graph of Figure 6 except that the errors instead of 
the signals themselves are shown. In Figure 7 the linear model error is represented by 
the lines with circles and the nonlinear model is represented by the dotted line. As can 

30 be readily appreciated, by observing the nonlinear model points at 701 A, B, and C and 
comparing it with the linear model points at 703 A, B, and C the nonlinear model 
achieves a significantly smaller error. 

Figure 8 is a graph representing the impulse response and a second order 
Volterra Kernel for a channel having an 850 nanometer wavelength VCSEL laser and 

35 utilizing fiber F0. Figure 8 illustrates an impulse response (the linear term of the overall 
response) at 801. Figure 8 also illustrates the response of the second order Volterra 
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Kernel at 803. In the present embodiment, the second order Volterra Kernel, which is 
associated with products of two consecutive bits, is the only significant nonlinear kernel. 
So the response of the second order Volterra Kernel generated from the product A N x 
A N . 1f i.e. FIR 513 in figure 5. FIR 511 corresponds to the linear kernel response 801 
and FIR 513 corresponds to the second order response 803. Using just the Volterra 
Kernels of FIR 511 and FIR 513 a close approximation of the channel which includes 
nonlinear elements and memory was achieved. The same method was used to 
examine the remaining fibers in the table of Figure 3. The signal-to-noise ratio for the 
linear model and for the nonlinear model (which was a linear model with nonlinear 
addition) was computed for each fiber. 

Signal-to-noise ratio is defined as 10 times the base 10 log of the signal power 
divided by the error power. To compute the signal-to-noise ratio the power for the 
signal was computed. The sum of the squares of the samples of the signal yields the 
total power of the signal and the sum of the squares of the samples of the error yields 
the error power. 

There are two versions of the error power, one of them for the linear model 
containing one Volterra Kernel and the other for the nonlinear model containing the 
additional nonlinear Volterra Kernels. The result of the error computations for the linear 
and the nonlinear model is illustrated in the table of figure 9. As can be seen from the 
table of figure 9 the signal-to-noise ratio for the linear model is in the range of 12 to 13 
dB. In the nonlinear model the error is significantly smaller, and correspondingly the 
signal to noise ratio is significantly higher in the range of 23 to 25 dB. The nonlinear 
model yields a model having approximately a 10 to 13 dB better signal to noise ratio, 
which is a significant improvement. 

The previously described measurements were all done at 850 nanometers 
wavelength. In the measurements taken, the nonlinearity was much larger at 850 
nanometers than at 1310 nanometers. If the same measurements are repeated at 
1310 nanometers, the linear model appears to be very adequate. 

Figure 10 is a graph of measured signal vs. linear model and the error between 
the measured signal and the linear model. Accordingly with the present data rate and 
fibers the linear channel model at 1 31 0 nanometers appears to be fairly accurate. The 
graph of figure 10 illustrates the measured signal as a solid line. The graph of figure 
10 is the result of a linear model shown as the line with circles. The line with diamonds 
represents the error. 

Figure 1 1 is a graph of the impulse response and a second order Volterra Kernel 
for the channel comprising fiber F0 and a DFB laser having a 1310 nano-meter 
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1 wavelength. The impulse response is illustrated at 1 1 01 and the second order Volterra 
Kernel is illustrated at 1 103. The second order Volterra Kernel is almost zero and most 
likely would not have to be taken into account in a model of the channel. 

Figure 12 is a table of signal-to-noise ratios for a linear channel model having a 

5 DFB laser operating at a wavelength of 1310 nanometers. All of the channel signal to 
noise ratios of all measured channels operating at a wavelength of 1310 nanometers 
were in the range of 26 to 28 dB indicating that a nonlinear term may not be necessary 
at a wavelength of 1310 nanometers, for a one gigahertz data rate. 

Additionally measurements were done including cases where mechanical motion 

10 of a fiber was introduced during the measurement process to examine motion induced 
nonstatioriarity of the general response. These observations for nonstationarity were 
done because there is some belief that the optical channel is not fixed in time and if 
vibration is introduced in the fiber some redistribution of the modes in the fiber may 
change the response of the fiber and accordingly the channel may be time-dependent. 

15 No evidence of any significant change in the fiber response as a result of motion was 
observed. 

Slow changes in the channel may be compensated by the LMS algorithm 
adjusting the channel model to track those types of changes. 

A program listing of a computer program used for the channel studies is included 
20 as Appendix A. Essentially, the program takes an input block of data from the 
measurements processes it, and then trains the canceller. It then computes the 
Volterra Kernels coefficients of the model using the linear model or the nonlinear model 
The program computes a signal to noise ratio and it prints out the results. 

Figure 13 is a block diagram of an optical receiver including a nonlinear optical 
25 channel equalizer. The nonlinear optical channel equalizer 1 300 accepts a signal which 
includes linear and nonlinear components. 

The electrical signal commonly results from converting han optical signal to the 
electrical domain using an optical to electrical converter 1307, such as for example a 
PIN diode, avalanche photodetector (APD), or another detector. The optical to 
30 electrical converter signal is amplified using a transimpedance amplifier (TIA) 1 309 and 
a postamplifier 1311. The postamplifier 1311 generally should not hard-limit a signal 
provided to it as hard limiting may introduce nonlinearity. 

After processing the signal provided to it the postamplifier 1311 may provide a 
processed signal to an optional analog processing block 1313. The analog processing 
35 block 1313 may provide such analog processing as filtering and automatic gain control, 
prior to providing an output signal to an analog to digital converter 1315. The output 
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signal from the analog to digital converter converts the electrical signal provided by the 
optional analog processing block 1313 to a multibit digital representation, prior to 
providing the multibit digital representation to a nonlinear optical channel equalizer 
1 300. The nonlinear optical equalizer 1 300 operates in the digital domain, and may be 
implemented using special-purpose digital logic or a general purpose programmable 
digital signal processor. Prior to entering the nonlinear optical equalizer 1300 the 
multibit signal may undergo additional preprocessing steps such as filtering, linear 
equalization, interference cancellation, etc. in an optional digital processing block 1317. 
Equalizer 1300 is a decision feedback equalizer comprising a detector such as slicer 
1303, a compensator such as subtracter 1301 and a feedback block 1305. The 
feedback block 1305 includes a nonlinear channel estimator employing a lookup table 
as illustrated in Figure 4 or the Volterra Kernel approach as illustrated in Figure 5. A 
difficulty with using the look up table approach of Figure 4 or the Volterra Kernel as 
illustrated in Figure 5 is that in both cases the Volterra Kernel or look up table 
coefficients need to be identified. One solution to the problem of coefficient 
identification is to have the transmitter send a training sequence, which is known a 
priori, to the receiver. The nonlinear channel estimator 1305 can then be trained using 
the known training sequence. In such a case, the functioning of the nonlinear channel 
estimator 1 305 would proceed as in Figure 4 or Figure 5 as the bit sequence would be 
known and differences produced by the channel could be readily identified. 

In practice, a training sequence may not be required because the decisions 1307 
of the decision feedback equalizer 1 300 and the nonlinear channel estimator 1 305 may 
still converge. Initially, because the channel estimator 1305 is not trained, the decisions 
1307 will have multiple errors. There is however, in most channels, a bias towards 
good decisions. Most decision feedback equalizers will converge if the channels do not 
exhibit excessive intersymbol interference (IS!)- Accordingly, if the coefficients in the 
nonlinear channel estimator 1305 are incorrect initially, there are both good and bad 
decisions. As long as the good decisions outnumber the bad decisions, there will be 
a converging trend and the nonlinear channel estimator 1305 will eventually tend to 
converge. 

The nonlinear channel estimator 1305 may also be trained using an LMS 
algorithm as illustrated in Figure 4 or 5. As the coefficients begin improving in the 
nonlinear channel estimator 1 305, the improvement helps make a larger percentage of 
correct decisions, which further assists in the convergence of the nonlinear channel 
estimating filter 1305. Once the values in the estimating filter converge, the nonlinear 
channel estimator 1 305 will have modified the coefficients so that essentially all correct 
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decisions are made. Any incorrect decisions made by the system tend to corrupt the 
values in the nonlinear channel estimator 1305, however, as long as the correct 
decisions out number the incorrect decisions the nonlinear channel estimator will 
converge to an essentially correct solution. In other words, as long as the majority of 
decisions are correct the coefficients of the nonlinear channel estimator 1 305 will have 
a net movement in the correct direction and converge. 

Once there is an initial convergence in the nonlinear channel estimator 1 305 the 
signal to the slicer 1303 is improved because at least some of the intersymbol 
interference is cancelled in summation unit 1301. As a result the number of correct 
decisions 1307 increases which leads to a better channel estimation. Past decisions 
are stored within the nonlinear channel estimator 1305 and are used to compute a 
replica of the intersymbol interference in order to provide it to summation unit 1301. 

If a Volterra Kernel structure similar to Figure 5 is used for the nonlinear channel 
estimator then the memory of past decisions are stored in the FIR filters 51 1 through 
517, which each nominal contain a FIR delay line structure. If the look up table 
approach is used then the memory for the past decisions are stored within a shift 
register, such as that illustrated in 407 at Figure 4. 

Figure 14A is a block diagram of an exemplary nonlinear channel equalizer as 
may be used in the nonlinear decision feedback equalizer of Figure 13. The nonlinear 
channel estimator Figure 14 is similar to the Volterra Kernel model illustrated in Figure 
5. The Volterra Kernel model of Figure 5 was used as an illustration of characterization 
of the channel. The nonlinear channel estimator illustrated in Figure 14 has an LMS 
update loop similar to Figure 5. Additionally, similar to Figure 5, the number of Volterra 
Kernels employed would depend upon the characteristics of the channel and the laser 
used to drive the channel. 

In Figure 14A, a signal 1427, which has been provided from an optical channel 
and converted to an electrical signal, is accepted by an optical channel equalizer, 
shown generally at 1400. The equalizer 1400 accepts the signal 1427jnto a summation 
unit 1301 where a (negated) correction signal 1429, representing the predicted inter- 
symbol interference, is added. Prior to the training and convergence of the Volterra 
Kernel estimator correction signal is zero or an arbitrary value. Slicer 1303 makes 
decisions 1 307 as to the values of the signal transmitted. Decisions 1 307 are accepted 
by Volterra Kernel estimator 1401 , which models the nonlinearities of the channel that 
provides the signal to the equalizer 1400. The output of the Volterra Kernel estimator 
provides a value of predicted intersymbol interference 1429, which can then be 
subtracted from the accepted signal in order to accurately recreate a transmitted signal 

-15- 



WO 02/43340 PCT/US01/43625 

1 which has been transmitted. 

The error in the predicted intersymbol interference 1429 is estimated by 
comparing decisions 1 307 from slicer 1 303 with the input signal 1422, which has been 
adjusted in summation unit 1301 by removing the predicted intersymbol interference ISI 

5 1429. The decisions 1307 and error 1425 are used to adjust the FIR filters 51 1 , 513, 
515, and 517, which exemplarily illustrate the FIR filters which comprise the Volterra 
Kernel estimator 1401. In such a manner, the Volterra Kernel estimator may be trained 
and compensate for non-linearities within the channel that provides the signal to the 
equalizer 1400. 

10 Figure 14B is an exemplary block diagram of a non-linear optical channel 

equalizer employing a lookup table channel estimator, according to the embodiment of 
the invention. In Figure 14 B an LMS algorithm may be employed in adjusting the 
values within the look up table 1435. The system 14B may be operated so that training 
of the equalizer is accomplished by processing of a signal provided from an optical 

15 channel, determining error signals and adjusting the equalizer 1401. In such a manner 
the equalizer may be trained to model the channel parameters. Those skilled in the art 
will recognize that the equalizers may be trained by employing a known sequence. 
Because such a sequence is known a priori the difference between the response of the 
channel and the desired response of the channel, i.e., the known sequence, can be 

20 readily determined. Such sequences may shorten the time necessary to train a 
nonlinear optical channel equalizer. 

A nonlinear channel estimator could also be built based on the look up table 
approach illustrated in Figure 4. The implementation of such a look up table version is 
a straight-forward adaptation of the channel characterization circuit illustrated in Figure 

25 4. 

Figure 15 is a block diagram of a nonlinear channel estimator combined with a 
Viterbi decoder. Instead of a decision feedback equalizer a Viterbi decoder may be 
used. Nonlinear channel estimator 1501 would be used to compute the expected signal 
going to the branch metric computations 1503. The remainder of the Viterbi decoder 

30 system including the actual Viterbi decoder itself, block 1505, is similar to the linear 
case. The nonlinear channel estimator 1501 is creating a model of the received signal. 
If the nonlinear channel estimator 1501 has a memory comprising k bits, that is it 
remembers the last k inputs, then the Viterbi decoder 1505 would have 2 (k1) states. 
Therefore, if k=3 then the Viterbi decoder 1505 would be a 4 state Viterbi decoder. If 

35 the nonlinear channel estimator 1501 has a memory of 3 bits then it can support eight 
different expected values of the received signal. Which would be implemented with a 
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1 look up table have eight locations. The expected values of the received signal are used 
by the branch metric computations 1 503 to compute all the branch metrics in the Viterbi 
decoder. 

The Viterbi decoder illustrated in Figure 1 5 is the same as a standard type Viterbi 
5 decoder except for the fact that the channel estimator function is a nonlinear channel 
estimator with memory. The nonlinearity is expressed either in the form of a look up 
table or a Volterra Kernel. 

The Viterbi decoder includes a receiver for receiving a signal including linear and 
non linear components. A non linear channel estimator computes the expected values 
10 of the received signal. A branch metrics computer computes the branch metrics based 
on the expected values of the received signal. A standard Viterbi decoder accepts the 
computed branch metrics and decodes the received signal. The non linear channel 
estimator can be fabricated using a Volterra kernel estimator or a lookup table estimator 
as illustrated at 1401 as illustrated at 1433 in Figure 14A or a look up table 
15 implementation as illustrated at 1433 in Figure 14B. 
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WHAT IS CLAIMED IS: 

1. A method for modeling the behavior of a data channel the method 
comprising: 

determining a sequence of data input to the data channel; 
using at least part of the sequence of data input to the data channel as an index 
to a channel model value; 

sampling the data after it has passed through the channel to produce a sampled 

value; 

comparing the channel model value with the sampled value; and 
adjusting the channel model value based on the results of the comparison 
between the channel model value and the sampled value. 

2. A method as in claim 1 wherein determining a sequence of data input to the 
data channel comprises determining the last N bits input to the channel. 

3. A method as in claim 2, where N = 5. 

4. A method as in claim 1 wherein the sampling of the data after it has passed 
through the channel to produce a sampled value comprises producing a real number 
representing the sampled value. 

5. A method as in claim 1 , wherein adjusting the channel model value further 
comprises adjusting the channel model value according to an LMS (Least Means 
Squared) algorithm. 

6. A method as in claim 1 wherein adjusting the channel model value further 
comprises adjusting the channel model value until it converges. 

7. A method as in claim 6 further comprising converting the look up table into 
Volterra Kernels. 

8. A method as in claim 7 further comprising converting the look up table into 
Volterra Kernels using a Hadamard transform. 

9. A method as in claim 7 further comprising adjusting the Volterra Kernels 
based on the results of the comparison between the channel model value and the 
sampled value. 
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10. A method as in claim 9 further comprising eliminating the insignificant 
Volterra Kernels. 

11. A method for modeling the behavior of a data channel the method 
comprising: 

determining a sequence of data input to the data channel; 
determining a Volterra Series representation of the channel; 
accepting at least part of the sequence of data input to the data into the Volterra 
series representation of the channel to produce a channel model value; 

sampling the data after it has passed through the channel to produce a sampled 

value; 

comparing the channei model value with the sampled value; and 
adjusting the channel model value based on the results of the comparison 
between the channel model value with the sampled value. 

12. A method as in claim 1 1 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 

accepting the most recent value of the sequence of data input to the data channel 
into a first FIR (Finite Impulse Response) filter; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and a second most recent value of the sequence of data input to the 
data channel into a second FIR; and 

summing an output of the first FIR and output of the second FIR to form the 
channel model value. 

13. A method as in claim 11 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 

accepting the most recent value of the sequence of data input to the data channel 
into a first FIR filter; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and the second most recent value of the sequence of data input to the 
data channel into a second FIR; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and a third most recent value of the sequence of data input to the data 
channel into a third FIR; and 

summing an output of the first FIR and output of the second FIR and output of the 
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third FIR to form the channel model value. 

14. A method as in claim 11 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 
accepting the most recent value of the sequence of data input to the data channel 
into a first FIR filter; 

accepting a product of the most recent value of the sequence of data input to the 
data channel and the second most recent value of the sequence of data input to the 
data channel into a second FIR; 

accepting a product of the most recent value of the sequence of data input to the 
data channel and a third most recent value of the sequence of data input to the data 
channel into a third FIR; 

accepting a product, said product being the most recent value of the sequence 
of data input to the data channel and the two next most recent data input, into a fourth 
FIR; and 

summing an output of the first FIR and output of the second FIR and output of the 
third FIR and output of the fourth FIR to form the channel model value. 

15. A method as in claim 1 1 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

16. A method as in claim 1 2 wherein the difference between thfe channel model 
value and the output of the channel is used to update all the FIRs. 

17. A method as in claim 1 3 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

18. A method as in claim 14 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

19. A method as in 15 wherein an LMS algorithm is used to update all the FIRs. 

20. A method as in 1 6 wherein an LMS algorithm is used to update all the FIRs. 

21 . A method as in 1 7 wherein an LMS algorithm is used to update all the FIRs. 
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22. A method as in 1 8 wherein an LMS algorithm is used to update all the FIRs. 

23. A method for equalizing an optical signal, modulated with a digital signal, 
received over an optical channel, the method comprising: 

converting the optical signal into an electrical signal; 
summing the electrical signal with a correction signal; 
providing the summed signal to a detector; 
detecting the summed signal to produce decisions; 
providing the decisions to a nonlinear channel estimator; and 
estimating the correction signal in the nonlinear channel estimator. 

24. The method of clam 23 wherein estimating the correction signal in the 
nonlinear estimator further comprises: 

accepting the decisions; 

predicting the inter-symbol interference of the channel in a nonlinear channel 
estimator; and 

forming a correction signal from the predicted inter-symbol interference. 

25. The method of claim 24 wherein predicting the inter-symbol interference 
of the channel further comprises: 

providing the decisions to a plurality of Volterra Kernels; and 

summing the output of the plurality Volterra Kernels to form a correction signal. 

26. The method of claim 25 further comprising: 

comparing the predicted inter-symbol interference to inter-symbol interference in 
the electrical signal; and 

updating the Volterra Kernels based on the result. 

27. The method as in claim 26 wherein updating the Volterra Kernels 
comprises using a LMS (Least Means Squared) algorithm to update the Volterra 
Kernels. 

28. The method of claim 24 wherein predicting the inter-symbol interference 
of the optical channel further comprises: 

providing the data decisions as an address into a look up table; 
qutputting a value stored in the look up table as the predicted inter-symbol 
interference; 

comparing the predicted inter-symbol interference to the inter-symbol interference 
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in the electrical signal; and 

updating the value stored in the look up table based on the result. 

29. The method as in claim 28 wherein updating the value stored in the look 
up table comprises using a LMS (Least Means Squared) algorithm. 

30. An apparatus for equalizing a signal received over an optical channel, the 
apparatus comprising: 

an input that accepts an optical signal; 

a converter that converts the optical signal into an electrical signal; 
a summation unit that sums the electrical signal with a correction signal; 
a detector that detects the summed signal to produce decisions; and 
a nonlinear channel estimator that estimates the correction signal. 

31. The apparatus of claim 30 wherein the nonlinear estimator further 
comprises: 

an input that accepts the decisions; 

an estimating circuit that predicts the inter-symbol interference of the channel; 

and 

an output that provides a correction signal from the estimated inter-symbol 
interference. 

32. The apparatus of claim 31 wherein estimating circuit that estimates the 
intersymbol interference of the channel further comprises: 

a plurality of Volterra kernels that accept the decisions and produce individual 
outputs; and 

a summation unit that sums the individual outputs of the plurality of Volterra 
Kernels to form a correction signal. 

33. The apparatus of claim 32 further comprising: 

a comparator for comparing the predicted inter-symbol interference to the inter- 
symbol interference in the electrical signal; and 

means for updating the Volterra kernels based on the result. 

34. The apparatuses in claim 33 wherein the comparator that compares the 
predicted inter-symbol interference to the inter-symbol interference in the electrical 
signal includes a LMS (Least Means Squared) algorithm that compares the predicted 
inter-symbol interference to the inter-symbol interference in the electrical signal. 
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35. The apparatus of claim 31 wherein the estimating circuit that estimates the 
intersymbol interference further comprises: 

a look up table which accepts the data decisions as an address into the look up 

table; 

values of the predicted inter-symbol interference stored in the look up table; 

a subtracter that subtracts the predicted inter-symbol interference from the 
electrical signal to produce a signal substantially free from intersymbol interference; 

a second subtractor that subtracts the decision from the substantially inter-symbol 
interference free signal to produce an error; 

and 

means for updating the value stored in the look up table based on the error. 

36. The apparatus as in claim 35 wherein comparing the comparator that 
compares the estimated inter-symbol interference to the inter-symbol interference in the 
electrical signal comprises a LMS (Least Means Squared) algorithm that compares the 
predicted inter-symbol interference to the inter-symbol interference in the electrical 
signal. 

37. A method for decoding a signal received over an optical channel, the 
method comprising: 

receiving a signal including linear and non linear components; 
estimating, in a non linear channel estimator, the expected values of the received 
signal; 

computing the branch metrics based on the expected values of the received 
signal; 

providing the computed branch metrics to a Viterbi decoder; and 
Viterbi decoding the received signal using the branch metrics provided to the 
Viterbi decoder. 

38. The method of claim 37 wherein estimating, in a noh linear channel 
estimator, the expected values of the received signal comprises: 

providing the value of the received signal to a Volterra kernel estimator; and 
computing the expected value sent based on the output of the Volterra kernel 
estimator. 

39. The method of claim 37 wherein estimating, in a non linear channel 
estimator, the expected values of the received signal comprises: 

providing the value of the received signal as an address to a look up table; and 
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looking up the stored value as the actual value transmitted . 

40. An apparatus for decoding a signal received over an optical channel, the 
apparatus comprising: 

a receiver for receiving a signal including linear and non linear components; 
a non linear channel estimator that computes the expected values of the received 
signal; 

a branch metrics computer for computing the branch metrics based on the 
expected values of the received signal; 

a Viterbi decoder that accepts the computed branch metrics and 
Viterbi decodes the received signal. 

41 . The apparatus of claim 40 wherein the non linear channel estimator that 
computes the expected values of the received signal comprises a Volterra kernel 
estimator that computes the expected value sent based on the output of the Volterra 
kernel estimator. 

42. The apparatus of claim 40 wherein the non linear channel estimator that 
computes the expected values of the received signal comprises a look up table that 
uses the value of the received signal as an address to look up the stored value as the 
actual value transmitted . 

43. A method for detecting digital data modulated on an optical signal and 
received over an optical channel, the method comprising: 

converting the optical signal to an electrical signal; 
converting the electrical signal to a multibit digital representation; 
estimating distortion introduced in the optical signal by the optical channel; 
compensating the multibit digital representation for the distortion; and 
detecting the digital data from the compensated multibit digital representation. 

44. A method as in claim 43 wherein estimating distortion introduced in the 
optical signal by the optical channel comprises estimating in a Volterra Kernel estimator 
the distortion introduced in the optical channel. 

45. A method as in claim 43 wherein estimating distortion introduced in the 
optical signal by the optical channel comprises estimating in a lookup table estimator 
the distortion introduced in the optical channel. 
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46. An apparatus for detecting digital data modulated on an optical signal and 
received over an optical channel/the apparatus comprising: 

a converter for converting the optical signal to an electrical signal; 
an analog to digital converter that converts the electrical signal to a multibit digital 
representation; 

an estimator that estimates distortion introduced in the optical signal by the 
optical channel; 

a compensator that compensates the multibit digital representation for the 
distortion; and 

a detector that detects the digital data from the compensated multibit digital 
representation. 

47. A method as in claim 45 wherein estimator that estimates distortion 
introduced in the optical signal comprises a Volterra Kernel estimator. 

48. A method as in claim 45 wherein estimator that estimates distortion 
introduced in the optical signal comprises a lookup table estimator. 

49. An apparatus as in claim 46 wherein the estimator that estimates distortion 
introduced in the optical signal comprises a Volterra Kernel estimator. 

50. An apparatus as in claim 46 wherein estimator that estimates distortion 
introduced in the optical signal comprises a lookup table estimator. 
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1 METHOD AND SYSTEM TO IDENTIFY AND CHARACTERIZE NONLINEARITIES 

IN OPTICAL COMMUNICATIONS CHANNELS 

FIELD OF THE INVENTION 
5 The invention relates to methods used to characterize communication channels 

and, in particular embodiments, to methods used to characterize and model nonlinear 
optical communication channels. 

BACKGROUND OF THE INVENTION 

10 Signal distortion in an optical channel may be caused by nonlinearities that exist 

in various components, which comprise an optical channel. Such distortion can be 
characterized by mathematical models of the nonlinear behavior of the optical 
communication channel. By using such models, compensatory measures, for example 
equalizers that compensate for distortion, which degrades the performance of the 

15 channel, may be designed. 

One source of distortion is dispersion. Dispersion may be a combination of linear 
and nonlinear components. 

Dispersion found in optical fibers, particularly multimode fibers, may limit the 
bandwidth in optical channels. The problem of dispersion may be more acute in 

20 multimode fibers in general, but it may also be a problem in single mode fibers, 
especially in long runs of fiber. Single mode fibers commonly have lower dispersion 
than multimode fiber. The dispersion commonly found in single mode fibers may be 
chromatic dispersion or polarization mode dispersion. The dispersion that is seen in 
single mode fibers is generally a lot less per length than the dispersion found in multi- 

25 mode fibers. Dispersion is cumulative, however, and so tends to increase as fibers 
increase in length. Accordingly, methods discussed herein in terms of multimode fibers 
may apply equally well to single mode fibers. 

In multi-mode fibers, the multiple modes of propagation within the fiber commonly 
cause dispersion. Generally, in the various modes of propagation within a fiber, the 

30 light signal has different speeds of propagation. So if a light pulse is sent from a 
transmitter, the pulse will propagate in multiple modes traveling at different speeds. 
The pulse, traveling in each mode, will reach the receiver at a different time. The 
multiple times of arrival of the input pulse traveling in several modes create a distorted 
pulse, somewhat like a spread out version of the transmitted pulse. 

35 Optical channels are inherently nonlinear. Electromagnetic fields can be 

described by Maxwell's equations, which are linear. Because Maxwell's equations are 



-1- 



WO 02/043340 



PCT/US01/43625 



1 linear, the principle of superposition applies to the electromagnetic field. What is 
modulated, however, in optical channels is not the electromagnetic field per se, it is the 
optical power. Superposition doesn't exactly apply to optical power. 

An optical photo detector is commonly a square law device. The photodetector 

5 responds linearly to the optical power, in fact the photo current generated by a 
photodetector is commonly a very accurately linearfunction of the optical power, but the 
power is a quadratic function of the electromagnetic field, thus creating a source of 
nonlinearity. 

It has been shown that if a laser has a large spectral width, instead of being a 
10 very monochromatic laser, the significant spectral width tends to result in a linearization 
of the channel even considering the effect of the optical power and the square law 
nature of the photodetector. In other words if a laser has a very monochromatic, narrow 
spectrum then the channel tends to behave less linearly than does a laser that has a 
wider spectrum. 

15 Nonlinear behavior in an optical channel may depend to varying extent on the 

properties of the photodetector and the fiber. Additionally there is a possibility of 
distortion from the laser itself. If there is dispersion in the laser, for example because 
the laser has some bandwidth limitation itself, then nonlinearity of the laser could also 
contribute to the total nonlinearity in the optical channel. 

20 When the intensity of an optical signal is high, a fiber optic which transmits the 

optical signal itself may introduce nonlinearities. These nonlinearities may result from 
the fact that the index of refraction of the fiber optic depends, to some extent, on the 
intensity of the optical signal itself. This effect commonly gives rise to nonlinearities 
known as "four-wave mixing", "self-phase modulation", and "cross-phase modulation". 

25 Additional nonlinearities may result from the phenomena known as stimulated Raman 
scattering and stimulated Brillouin scattering. For a more comprehensive treatment of 
nonlinearities, see the text "Fiber-Optic Communication Systems", second edition, by 
Govind P. Agrawal, John Wiley and Sons, 1997, ISBN 0-471-17540-4, which is 
incorporated by reference herein. 

30 Although the above sources of nonlinearity in an optical channel are presented 

as examples, other sources may exist. The techniques described in this disclosure may 
be applied to all kinds of nonlinearities regardless of their physical source, as will be 
clear to those skilled in the art. 

35 SUMMARY OF EMBODIMENTS OF THE INVENTION 

In one aspect of the invention, the disclosure illustrates a method for modeling 
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1 the behavior of a data channel. To create the model, the sequence of data Input to the 
data channel is examined. A portion of the sequence of data input to the data channel 
is used as an index to a channel model value. Output data from the channel is 
sampled. A numerical channel model value is compared with the sampled value and 

5 the channel model value is adjusted based on the results of the comparison. 

In another aspect of the invention, a method for modeling the behavior of a data 
channel using Volterra Kernels is illustrated. A sequence of data is provided to the data 
channel. A part of the sequence of data input to the channel is provided to the Volterra 
Kernel model of the channel. The VoJ|erra Kernel (VK) model produces a channel 

10 model value. The data values measured from the channel are compared to the VK 
channel model value and the results used to update the Volterra Kernel model. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The features, aspects, and advantages of the present invention, which have been 
15 described in the above summary, will be better understood with regard to the following 
description, appended claims, and accompanying drawings where: 

Figure 1 is a graphical illustration of an environment in which embodiments of the 
present invention may be used. 

Figure 2 is a graphical illustration of an equipment setup, which may be used to 
20 characterize optical channels. 

Figure 3 is a table listing of fibers, which were measured using the measurement 
setup illustrated in figure 2. 

Figure 4 is a graphic illustration of an example of a system that may be used for 
channel identification. 

25 Figure 5 is a graphical illustration of channel identification based on Volterra 

Kernels (VK). 

Figure 6 is a graph illustrating measurement of a channel having an 850 
nanometer VCSEL (Vertical Cavity Surface Emitting Laser) and fiber F0, and 
comparing the signal measurements to linear and nonlinear channel models. 

30 Figure 7 is a graph illustrating the measurement of a channel having an 850 

nanometer VCSEL and fiber F0, and comparing the amount of error generated by 
linear and nonlinear channel models. 

Figure 8 is a graph representing an impulse response and a second order 
Volterra Kernel for an 850 nanometer VCSEL and fiber F0. 

35 Figure 9 is a table of the signal to noise ratios computed for linear and nonlinear 

channel models for a channel operated at a wavelength of 850 nanometers. 
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Figure 10 is a graph comparing linear model, measured signal and error within 
a channel using a fiber F0 at a wavelength of 131 0 nanometers. 

Figure 11 is a graph of the impulse response and a second order Volterra Kernel 
for a channel using fiber F0 and a distributed feedback (DFB) laser with 1310 
nanometer wavelength. 

Figure 12 is a table of signal-to-noise ratios for a linear model of a channel 
employingl 310 nanometer DFB laser. 

Figure 13 is a block diagram of a nonlinear optical channel equalizer, according 
to an embodiment of the invention. & 

Figure 14A is a block diagram of a nonlinear optical channel equalizer that 
employs a Volterra Kernel estimator according to an embodiment of the invention. 

Figure 14B is a block diagram of a non-linear optical channel equalizer employing 
a lookup table channel estimator, according to an embodiment of the invention. 

Figure 1 5 is a Viterbi decoder employing a nonlinear channel estimator, according 
to an embodiment of the invention. 

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION 

The listed figures describe and illustrate various aspects of the present invention. 
Accordingly, Figure 1 is a graphic illustration of an environment in which 

embodiments of the present invention may be used. The environment illustrated at 1 01 

is an information distribution system, such as may be found, for example, in a computer 

network accessing a remote database. 

In Figure 1, users 105 are coupled to a server 103. The users then may 

communicate with remote database 107 using fiber optic 109. Fiber optics may be 

used to communicate large amounts of data between points because of their large 

bandwidth. 

The data rate of optical channels is, in part, limited by nonlinearities in the 
channel. The term "channel", as used herein, includes the laser, amplifiers that drive 
the laser, the fiber optic, the photo detector receiver and all elements included in the 
transmission and reception of a fiber optic signal. 

For characterizing optical channels, the measurement setup illustrated in Figure 

2 was used. 

Figure 2 is a graphical illustration of an equipment setup used to characterize 
optical channels. A signal is generated by modulating a laser 219. The modulated 
laser signal is provided to a fiber optic 215. In the embodiment illustrated in Figure 2, 
the signal provided to the optical channel was a 1 Gigahertz (GHz), 127 bit, pseudo- 
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random binary sequence provided by bit error rate tester 201 . The resulting received 
signal was detected by a photodetector 207, captured using an oscilloscope 205 and 
stored as a file in computer 209. Using computer 209, the received signal may be 
processed and the nonlinearity that exists in the received signal can be identified. The 
approach taken is that no attempt is made to identify the source of the nonlinearity as 
belonging to a specific channel component such as a photodetector or laser or any 
other source. Instead the nonlinearity in the received signal was examined without 
being concerned for the physical source of the nonlinearity. 

Two types of lasers were tested. The first type was a 850 nanometer VCSEL 
(vertical cavity surface emitting laser) and the second type of laser was a 1310 
nanometer DFB (distributed feedback) laser. All measurements were done at a data 
rate of 1 gigabit per second. 

The bit error rate data tester 201 generated a pseudo random binary sequence 
that was periodically repeated. The channel measurements were performed at a data 
rate of 1 gigabit per second, but the method may be applied to any data rate. 

In the setup illustrated in Figure 2 free space optics and an optical bench 221 
were used. The laser was aligned using alignment screws so that the laser 219, the 
fiber 215 and the lens were arranged to focus the light into fiber 215. Laser 219 
requires a DC bias current provided by bias power supply 203. A bias Tee 21 1 allows 
the combination of the data signal with a bias in order to apply a correctly biased data 
signal to the laser 219. 

A length of fiber 215 is contained in a spool so that lengthy fibers up to more than 
1 kilometer may be used. In the case of single-mode fibers, the length of the spool 
could exceed 700 km. Photodetector 209 is a commercial unit having a bandwidth of 
1 0 gigahertz. The oscilloscope 205 is a real time sampling oscilloscope which samples 
at 8 gigahertz. Because the data rate used was 1 gigabit per second, the received 
signal was sampled on average 8 times per bit. Samples of the received signal were 
captured in the memory of the oscilloscope and then downloaded to the computer 209 
in the form of a file. Typical received signal data files contained about 65,000 data 
points representing received samples of the pseudo random input sequences of 127 
bits. 

Once the data points have been accumulated they may be processed and 
information about the behavior of the optical channel, particularly the nonlinearity, may 
be extracted. Additionally any possible nonstationarity of the channel may be identified. 
In other words, if the channel changes in time, such nonstationarity may be detected. 
In the measurements taken, however, no significant nonstationarity was observed. 
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1 Figure 3 is a table listing of fibers, which were measured using the measurement 

setup illustrated in figure 2. Six different fibers, having core diameters from 50 to 62-1/2 
microns and the lengths from 270 meters to about 2.2 kilometers, were tested. All of 
the fibers exhibit some dispersion phenomenon and bandwidth limitation to varying 

5 extent. 

To model channel nonlinearity two different mathematical methods, representing 
nonlinear functions having memory, were chosen. Because dispersion is inherently 
a function having memory, the resulting channel nonlinearity is a nonlinear function with 
memory, as opposed to memoryless nonlinearity. Memoryless nonlinearity may be 

10 represented by a function of the form Y = F(X), with the nonlinearity an arbitrary 
nonlinear function F of a single variable. Nonlinearity with memory can be described 
as a nonlinear function of a number of consecutive samples of the transmitted data, so 
that the nonlinear function with memory is of the form Y = F( X n ,X n . 1> X fv2> ..). In other 
words nonlinearity with memory is a function of several variables, where the variables 

1 5 are exemplarily samples of the transmitted bit sequence. Nonlinearity with memory is 
a more general kind of nonlinearity that may be found in optical channels. In other 
words the characteristic of an optical channel is not only a function of the current bit, but 
depends also on previously transmitted bits. 

The nonlinearity in an optical channel can be modeled as a look-up table, for 

20 example as a nonlinear function of the five most recently transmitted bits. In a digital 
transmission system the transmitted bits have 2 levels, 0 and 1 (or+1 and-1 depending 
on the convention that is used). Because of the discrete nature of the transmitted bits, 
the function characterizing a channel depending on the five most recent bits can only 
have 2 5 =32 values. Accordingly there are 32 possible values that model the channel 

25 by listing all the values for all the possible sequences of the most recent 5 input bits. 
Such a model can be implemented as a look-up table. The look up table is one model 
with memory for characterizing an optical channel having nonlinearity. 

A second optical channel model is based on a Volterra series. Volterra series 
have been studied in mathematics for a considerable time. Volterra series are a way 

30 to represent nonlinear functions. In general Volterra series are infinite series, i.e. a 
series with infinite number of terms. In the present case, however, since the transmitted 
data is binary, the number of terms is finite. The number of terms in the series used to 
model a binary optical channel is 2 to the power N, where N is the number of most 
recent transmitted bits, which are taken into account. That is the same number of terms 

35 as the number of entries in a look-up table. Accordingly, no matter what description is 
used, whether the.look-up table approach or the Volterra series approach, the number 
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1 of required elements in each model is 2 to the power N, where N is the number of most 
recent bits transmitted that are taken into account. In the look-up table approach the 
2 to the N parameters are the values of the channel for all the possible values of the 
most recent N input bits. 

5 For the Volterra series, the parameters are the coefficients of the series and 

because the series has 2 N terms there are 2 N coefficients. 2 N is the same number of 
parameters as required by the look up table model. Accordingly regardless of which 
description is chosen, the maximum number of parameters that is needed to completely 
specify the nonlinear function is the same, i.e. 2 N . It can be shown that the Volterra 

10 Kernel representation is mathematically equivalent to the look-up table. In fact a 
mathematical transformation known as the Hadamard transformation can transform the 
look up table model to the Volterra Kernel model and Volterra Kernel model to the look 
up table model. 

N is generally an empirically determined value, the exemplary value of N=5 in the 

15 present embodiment appears to work very well. The value of N, of course could 
depend on the fibers, wavelength of the laser and the rate of the data being transmitted. 
In principle it is possible to find fibers where N is larger or smaller, but in present 
embodiments N=5 is sufficient. 

An advantage of choosing the Volterra series description is that there is a way to 

20 make the description somewhat compact by grouping terms. By grouping the terms 2 N 
terms are transformed into 2 (N1) convolution-like terms having a shift register-type 
sequence. That means, for example, the terms that go into the sequence of grouped 
terms are products of bits. So, for example, a X N term would be multiplied by an X N-1 
term and so on. The shift register sequence of grouped terms means shifted versions 

25 of the product. So, for example, this shift register sequence would be in the form X N * 
x n-i» *n-i * x n-2. x n -2 * x n-3 and so on - This is shift register-type sequence is one that 
can describe terms of the Volterra series as a convolution of this kind of shift register- 
type Sequence with a certain kernel. A kernel is an element that resembles an impulse 
response. A kernel is not an impulse response in the traditional sense, but a traditional 

30 impulse response is used as a linear portion of the nonlinear kernel function. 

Terms involving a convolution of a shift register type sequence with a certain 
kernel, are referred to as Volterra Kernels. There are multiple Volterra Kernels 
corresponding to different kinds of products of bits. The maximum number of Volterra 
Kernels needed to represent an N bit sequence to is 2 N ~ 1 . So, for example, in the case 

35 ' where N=5, the total number of terms in the Volterra series is 2 N or 32 terms. But these 
32 terms can be mapped into 1 6 Volterra Kernels. Sixteen Volterra Kernels suffice and 
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the number of Volterra Kernels is equal to 2 (N1) , or in the present example 2 4 or 16. So 
there are 16 Volterra Kernels and the Volterra Kernels may provide a more convenient 
and more compact description of a nonlinearity particularly because commonly in many 
applications the number of Volterra Kernels is actually less than 2 (N Although the 
number of Volterra Kernels to theoretically define a Volterra Kernel model is 2 (N - 1) , some 
of the kernels are very small and can be neglected. In some situations the channel may 
be effectively modeled by one or two Volterra Kernels. That is the situation in the 
optical channels of the present embodiment. In present embodiments the nonlinearity 
is simple enough so that it can be described completely by a linear term and a single 
Volterra Kernel, indicating that a fairly simple kind of nonlinearity is present. 

Because the nonlinearity is simple doesn't mean that its negligible. In fact in 
present embodiments the nonlinearity, though simple, is a very strong. Because the 
nonlinearity has a simple description, simple equalizers may be built. In other words 
nonlinear equalizers with memory which are relatively simple but very powerful may be 
built. Such equalizers can be very effective because they can deal with the nonlinearity 
that is strong whereas a traditional kind of linear equalizer might fail completely. 

Figure 4 is a graphic illustration of an example of a system that may be used for 
channel identification. The system is implemented in software. 

Input bits 401 are provided to a channel 403. What is used for the channel 403 
is the measurements that were taken with the setup illustrated in Figure 2. So the 
channel input and channel response are actual measured values. 

In the present embodiment the measurements are stored in a file and then 
processed off-line in a computer using a software program that implements the 
remainder of the system illustrated in Figure 4. This was done for the sake of 
convenience because the equipment for the setup in Figure 2 was readily available. 
There is however no constraint to prevent the present disclosed methods from being 
implemented as part of a real time or embedded system. 

The look-up table 409 is adaptive. That is the value in it can be adapted using 
the LMS (least-means-squared) algorithm. Basically what is done is to compare the 
measured sequence from the channel 403 (the file of received values stored by 
computer 209) with the output of the look-up table. The addresses in the look up table 
409 are received from a shift register 407 representing the values of the latest 5 input 
bits provided as input to the channel. Initially the look-up table values are initialized to 
all zeros started from situations where there is no knowledge of the nonlinearity. In 
such a case the error will be equal to the input signal. So the error will be large initially. 
The values in the look up table can be adjusted using an LMS algorithm to update the 
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1 look-up table according to the error 405 observed. The LMS algorithm will converge so 
that it tends to drive the error to zero or as close as possible to zero. 

Once the look-up table converges the error will be small or zero. In actual 
practice the error may not be exactly zero because the input signal may include some 

5 random noise. Additionally it is also possible the look-up table doesn't perfectly model 
all the nonlihearity. There may be some residual unaccounted for nonlinearity because 
N was chosen equal to 5 but in practice a N of 8 or 1 6 may have been required to more 
closely model the fiber. The choice of N is an implementation detail depending on the 
type of fiber, data rate, and a variety of other factors. 

10 Additionally in figure 4 the look up table is like having an array where the address 

is created by looking at 5 consecutive bits, but the number is accessed is a single 
number that could be represented by multiple bits which could be 16 bits or 24 bits, and 
as such may contain some quantization error. 

Figure 5 is a graphical example of channel identification based on Volterra 

15 Kernels. The Volterra Kernel approach is an approach mathematically equivalent to the 
use of a look-up table, as illustrated in Figure 4. In Figure 5, four Volterra Kernels are 
used. Normally there would be 16 Volterra Kernels present in a case where N is equal 
to 5. In some cases some of the Volterra Kernels may be negligible and depending on 
the Bit Error Rate (BER) required only the most important ones may need to be taken 

20 into account in the model, as for instance illustrated in Figure 5. For example in Figure 
5 there is a linear term which is represented by the Kernel labeled FIR 511. "FIR" 
indicates that each Kernel is represented by a finite impulse response (FIR) filter. In 
Figure 5, FIR 51 1 directly takes a sequence of input bits and creates a convolution of 
the input sequence with an impulse response of FIR filter 511. Hence FIR 51 1 is a 

25 linear term, i.e. a traditional convolution. So if only a term represented by FIR 51 1 is 
needed for the channel model, the channel is essentially linear. If, in addition to FIR 
51 1 , other terms are present then the model is nonlinear. 

FIR 51 3 is a Volterra Kernel which is associated with products of two consecutive 
bits, i.e. a second order Volterra Kernel. FIR 513 is a second order Volterra Kernel 

30 because it accepts a term 505 which is a product of the most recent input bit and the 
bit preceding the most recent input bit. The most recent input bit is designated as A N 
and the bit preceding the most recent bit is designated A N . r 

Second order Volterra Kernel FIR 513 is associated with two (consecutive) bits 
i.e. bits that are one time delay apart/it accepts the product of two (consecutive) bits. 

35 Another second order Volterra Kernel 51 5 is produced by taking the product of bits that 
are two time delays apart, 501 and 503 representing time delays. The most recent bit 
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and the bit prior to the second most recent bit are multiplied and result in output 507, 
which is the accepted by FIR 51 5. In other words A N is multiplied by A N _ 2 , here, because 
this is delayed by 2 units. FIR 515 is a Finite Impulse Response filter Gust as FIR 511 
and FIR 51 3) but the input, instead of being bits is the product of 2 bits, and so it is a 
second order Volterra Kernel. FIR 515 accepts the product of bits A N and A N . 2 , so it is 
a different type of Volterra Kernel than FIR 513 that accepts sequential bits. Both FIR 
515 and FIR 51 3 are second order. 

A third order Volterra Kernel FIR 517 is determined by the product (509) of 3 bits. 
509 is a product of 3 consecutive bits A N , A N1 and A N . 2 . Accordingly FIR 517 is a third 
order Volterra Kernel. 

In the simplified model illustrated in Figure 5 only the aforementioned 4 kernels, 
the linear kernel, two second orders kernels, and a third order kernel are present. In 
practice, if a nonlinearity were very strong, the Volterra Kernel model could have 16 
kernels or more if N greater than 5 (where N is the number of most recently transmitted 
bits to be considered) . In the present embodiment out of the 16 Volterra Kernels only 
the four illustrated in Fig. 5 are significant, the others are negligible. 

In the present embodiment FIR 51 1 and FIR 51 3 are much more significant than 
FIR 515 and FIR 517, and in fact FIR 515 and FIR 517 could be completely neglected. 

The Volterra Kernel model tends to simplify the description of the channel 
because it reduces the number of independent terms that need to be taken into 
account. Volterra Kernels may group together all the terms that are time-shifted 
versions of each other. In the more general Volterra Series description, all the terms 
are represented independently. . When groups of terms of a Volterra Series are time- 
shifted versions of each other, they may be associated with each other, and all the ones 
that are time-shifted versions of each other may be grouped as a single Volterra Kernel. 

The two equivalent representations - the look-up table model and the Volterra 
Kernel model are mathematically equivalent because the Hadamard transformation can 
transform one into the other. The reason to choose one or the other is essentially ease 
of implementation. The model that is simpler is commonly dependent on the particular 
problem that is trying to be solved. For example, if a strongly nonlinear function with 
short memory is present (meaning small amounts of dispersion) the look-up table 
probably provides a simpler model. The look up table is likely simpler because, if the 
dispersion is not large N will not be large. If N is not large then 2 N is not a very large 
number of coefficients. But if the memory span of the nonlinearity (i.e. N) grows then 
the look-up table grows exponentially because the size of the look-up table is 2 N . The 
size of the look up table may become prohibitively complex for large N . In some cases 
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1 Volterra Kernels may provide a simpler model when N is large and the nonlinearity has 
a simple form which can be described with few kernels, such as the previous example 
that could be limited to two kernels. 

If N is large, it will generally indicate that there is a lot of dispersion in the channel, 

5 but the form of the nonlinear function could be very simple. The function may be simple 
enough that it can be described, as in the previous example, by only two Volterra 
Kernels. If N is large and the Volterra model is simpler, the nonlinearity may be 
completely modeled with two kernels, even though N is very large. Accordingly in some 
cases the Volterra Kernel description is simpler. In other cases the look-up table 

10 description is simpler. 

In the studies performed, the amount of dispersion necessitated an N on the 
order of 5, but in principle, for severely bandwidth limited fibers, N could be larger. In 
the case where two Volterra Kernels are sufficient to create an accurate channel model, 
regardless of how large N is, the Volterra Kernel description is generally simpler. 

15 Figure 6 is a graph illustrating channel measurement, of a channel having an 850 

nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
linear and nonlinear fiber models. In Figure 6 the solid line corresponds to the 
observations, the dotted line represents the linear model, and the dashed and dotted 
represents the nonlinear model. As can be seen from 601, 603 and 605 the nonlinear 

20 model approximates the measurements of the channel to a greater degree than the 
linear model. Accordingly if the channel model were limited to a linear model a good 
fit for the transmitted data would not be achieved. In contrast using the nonlinear model 
a significantly closer fit may be obtained. 

Figure 7 is a graph illustrating channel measurement of a channel having an 850 

25 nanometer VCSEL laser and fiber F0, and comparing the signal measurement with 
amount of error generated by linear and nonlinear channel models. Figure 7 is a graph 
representing the same time as the graph of Figure 6 except that the errors instead of 
the signals themselves are shown. In Figure 7 the linear model error is represented by 
the lines with circles and the nonlinear model is represented by the dotted line. As can 

30 be readily appreciated, by observing the nonlinear model points at 701 A, B, and C and 
comparing it with the linear model points at 703 A, B, and C the nonlinear model 
achieves a significantly smaller error. 

Figure 8 is a graph representing the impulse response and a second order 
Volterra Kernel for a channel having an 850 nanometer wavelength VCSEL laser and 

35 utilizing fiber F0. Figure 8 illustrates an impulse response (the linear term of the overall 
response) at 801. Figure 8 also illustrates the response of the second order Volterra 
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Kernel at 803 In the present embodiment, the second order Volterra Kernel, which is 
associated with products of two consecutive bits, is the only significant nonlinear kernel. 
So the response of the second order Volterra Kernel generated from the product A N x 
A ie FIR 513 in figure 5. FIR 511 corresponds to the linear kernel response 801 
and FIR 51 3 corresponds to the second order response 803. Using just the Volterra 
Kernels of FIR 511 and FIR 513 a close approximation of the channel which includes 
nonlinear elements and memory was achieved. The same method was used to 
examine the remaining fibers in the table of Figure 3. The signal-to-noise rat.o for the 
linear model and for the nonlinear model (which was a linear model with nonl.near 
addition) was computed for each fiber. 

Signal-to-noise ratio is defined as 10 times the base 10 log of the signal power 
divided by the error power. To compute the signal-to-noise ratio the power for the 
signal was computed. The sum of the squares of the samples of the signal yields the 
total power of the signal and the sum of the squares of the samples of the error yields 
the error power. 

There are two versions of the error power, one of them for the l.near model 
containing one Volterra Kernel and the other for the nonlinear model conta.n.ng the 
additional nonlinear Volterra Kernels. The result of the error computations for the l.near 
and the nonlinear model is illustrated in the table of figure 9. As can be seen from the 
table of figure 9 the signal-to-noise ratio for the linear model is in the range of 12 to 13 
dB In the nonlinear model the error is significantly smaller, and correspondingly the 
signal to noise ratio is significantly higher in the range of 23 to 25 dB. The nonlinear 
model yields a model having approximately a 10 to 13 dB better signal to noise rafio, 
which is a significant improvement. 

The previously described measurements were all done at 850 nanometers 
wavelength. In the measurements taken, the nonlinearity was much larger at 850 
nanometers than at 1310 nanometers. If the same measurements are repeated at 
1310 nanometers, the linear model appears to be very adequate. 

Figure 10 is a graph of measured signal vs. linear model and., the error between 
the measured signal and the linear model. Accordingly with the present data rate and 
fibers the linear channel model at 1310 nanometers appears to be fairly accurate^ The 
graph of figure 1 0 illustrates the measured signal as a solid line. The graph of figure 
1 0 is the result of a linear model shown as the line with circles. The line with diamonds 

represents the error. 

FigureH is a graph ofthe impulse response and a second order Volterra Kernel 
for the channel comprising fiber F0 and a DFB laser having a 1310 nano-meter 



WO 02/043340 



PCT/US01/43625 



wavelength. The impulse response is illustrated at 1 101 and the second order Volterra 
Kernel is illustrated at 1 103. The second order Volterra Kernel is almost zero and most 
likely would not have to be taken into account in a model of the channel. 

Figure 12 is a table of signal-to-noise ratios for a linear channel model having a 
DFB laser operating at a wavelength of 1310 nanometers. All of the channel signal to 
noise ratios of all measured channels operating at a wavelength of 1310 nanometers 
were in the range of 26 to 28 dB indicating that a nonlinear term may not be necessary 
at a wavelength of 1310 nanometers, for a one gigahertz data rate. 

Additionally measurements were done including cases where mechanical motion 
of a fiber was introduced during the measurement process to examine motion induced 
nonstationarity of the general response. These observations for nonstationarity were 
done because there is some belief that the optical channel is not fixed in time and if 
vibration is introduced in the fiber some redistribution of the modes in the fiber may 
change the response of the fiber and accordingly the channel may be time-dependent. 
No evidence of any significant change in the fiber response as a result of motion was 
observed. 

Slow changes in the channel may be compensated by the LMS algorithm 
adjusting the channel model to track those types of changes. 

A program listing of a computer program used for the channel studies is included 
as Appendix A. Essentially, the program takes an input block of data from the 
measurements processes it, and then trains the canceller. It then computes the 
Volterra Kernels coefficients of the model using the linear model or the nonlinear model 
The program computes a signal to noise ratio and it prints out the results. 

Figure 13 is a block diagram of an optical receiver including a nonlinear optical 
channel equalizer. The nonlinear optical channel equalizer 1 300 accepts a signal which 
includes linear and nonlinear components. 

The electrical signal commonly results from converting han optical signal to the 
electrical domain using an optical to electrical converter 1307, such as for example a 
PIN diode, avalanche photodetector (APD), or another detector. The optical to 
electrical converter signal is amplified using a transimpedance amplifier (TIA) 1 309 and 
a postamplifier 1311. The postamplifier 1311 generally should not hard-limit a signal 
provided to it as hard limiting may introduce nonlinearity. 

After processing the signal provided to it the postamplifier 1311 may provide a 
processed signal to an optional analog processing block 1313. The analog processing 
block 1313 may provide such analog processing as filtering and automatic gain control, 
prior to providing an output signal to an analog to digital converter 1315. The output 
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signal from the analog to digital converter converts the electrical signal provided by the 
optional analog processing block 1313 to a multibit digital representation, prior to 
providing the multibit digital representation to a nonlinear optical channel equalizer 
1 300. The nonlinear optical equalizer 1 300 operates in the digital domain, and may be 
implemented using special-purpose digital logic or a general purpose programmable 
digital signal processor. Prior to entering the nonlinear optical equalizer 1300 the 
multibit signal may undergo additional preprocessing steps such as filtering, linear 
equalization, interference cancellation, etc. in an optional digital processing block 1 317. 
Equalizer 1300 is a decision feedback equalizer comprising a detector such as slicer 
1303, a compensator such as subtracter 1301 and a feedback block 1305. The 
feedback block 1305 includes a nonlinear channel estimator employing a lookup table 
as illustrated in Figure 4 or the Volterra Kernel approach as illustrated in Figure 5. A 
difficulty with using the look up table approach of Figure 4 or the Volterra Kernel as 
illustrated in Figure 5 is that in both cases the Volterra Kernel or look up table 
coefficients need to be identified. One solution to the problem of coefficient 
identification is to have the transmitter send a training sequence, which is known a 
priori, to the receiver. The nonlinear channel estimator 1305 can then be trained using 
the known training sequence. In such a case, the functioning of the nonlinear channel 
estimator 1305 would proceed as in Figure 4 or Figure 5 as the bit sequence would be 
known and differences produced by the channel could be readily identified. 

In practice, a training sequence may not be required because the decisions 1 307 
of the decision feedback equalizer 1 300 and the nonlinear channel estimator 1 305 may 
still converge. Initially, because the channel estimator 1 305 is not trained , the decisions 
1307 will have multiple errors. There is however, in most channels, a bias towards 
good decisions. Most decision feedback equalizers will converge if the channels do not 
exhibit excessive intersymbol interference (ISI). Accordingly, if the coefficients in the 
nonlinear channel estimator 1305 are incorrect initially, there are both good and bad 
decisions. As long as the good decisions outnumber the bad decisions, there will be 
a converging trend and the nonlinear channel estimator 1305 will eventually tend to 
converge. 

The nonlinear channel estimator 1305 may also be trained using an LMS 
algorithm as illustrated in Figure 4 or 5. As the coefficients begin improving in the 
nonlinear channel estimator 1 305, the improvement helps make a larger percentage of 
correct decisions, which further assists in the convergence of the nonlinear channel 
estimating filter 1305. Once the values in the estimating filter converge, the nonlinear 
channel estimator 1 305 will have modified the coefficients so that essentially all correct 
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1 decisions are made. Any incorrect decisions made by the system tend to corrupt the 
values in the nonlinear channel estimator 1305, however, as long as the correct 
decisions out number the incorrect decisions the nonlinear channel estimator will 
converge to an essentially correct solution. In other words, as long as the majority of 

5 decisions are correct the coefficients of the nonlinear channel estimator 1 305 will have 
a net movement in the correct direction and converge. 

Once there is an initial convergence in the nonlinear channel estimator 1 305 the 
signal to the slicer 1303 is improved because at least some of the intersymbol 
interference is cancelled in summation unit 1301. As a result the number of correct 

10 decisions 1307 increases which leads to a better channel estimation. Past decisions 
are stored within the nonlinear channel estimator 1305 and are used to compute a 
replica of the intersymbol interference in order to provide it to summation unit 1301. 

If a Volterra Kernel structure similar to Figure 5 is used for the nonlinear channel 
estimator then the memory of past decisions are stored in the FIR filters 51 1 through 

15 517, which each nominal contain a FIR delay line structure. If the look up table 
approach is used then the memory for the past decisions are stored within a shift 
register, such as that illustrated in 407 at Figure 4. 

Figure 14A is a block diagram of an exemplary nonlinear channel equalizer as 
may be used in the nonlinear decision feedback equalizer of Figure 13. The nonlinear 

20 channel estimator Figure 14 is similar to the Volterra Kernel model illustrated in Figure 
5. The Volterra Kernel model of Figure 5 was used as an illustration of characterization 
of the channel. The nonlinear channel estimator illustrated in Figure 14 has an LMS 
update loop similar to Figure 5. Additionally, similar to Figure 5, the number of Volterra 
Kernels employed would depend upon the characteristics of the channel and the laser 

25 used to drive the channel. 

In Figure 14A, a signal 1427, which has been provided from an optical channel 
and converted to an electrical signal, is accepted by an optical channel equalizer, 
shown generally at 1400. The equalizer 1400 accepts the signal 1427 into a summation 
unit 1301 where a (negated) correction signal 1429, representing the predicted inter- 

30 symbol interference, is added. Prior to the training and convergence of the Volterra 
Kernel estimator correction signal is zero or an arbitrary value. Slicer 1303 makes 
decisions 1 307 as to the values of the signal transmitted. Decisions 1307 are accepted 
by Volterra Kernel estimator 1401 , which models the nonlinearities of the channel that 
provides the signal to the equalizer 1400. The output of the Volterra Kernel estimator 

35 provides a value of predicted intersymbol interference 1429, which can then be 
subtracted from the accepted signal in order to accurately recreate a transmitted signal 
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which has been transmitted. 

The error in the predicted intersymbol interference 1429 is estimated by 
comparing decisions 1 307 from slicer 1 303 with the input signal 1 422, which has been 
adjusted in summation unit 1301 by removing the predicted intersymbol interference ISI 
1429. The decisions 1307 and error 1425 are used to adjust the FIR filters 51 1 , 513, 
515, and 517, which exemplarily illustrate the FIR filters which comprise the Volterra 
Kernel estimator 1401. In such a manner, the Volterra Kernel estimator may be trained 
and compensate for non-linearities within the channel that provides the signal to the 
equalizer 1400. 

Figure 14B is an exemplary block diagram of a non-linear optical channel 
equalizer employing a lookup table channel estimator, according to the embodiment of 
the invention. In Figure 14 B an LMS algorithm may be employed in adjusting the 
values within the look up table 1 435. The system 14B may be operated so that training 
of the equalizer is accomplished by processing of a signal provided from an optical 
channel, determining error signals and adjusting the equalizer 1401 . |n such a manner 
the equalizer may be trained to model the channel parameters. Those skilled in the art 
will recognize that the equalizers may be trained by employing a known sequence. 
Because such a sequence is known a priori the difference between the response of the 
channel and the desired response of the channel, i.e., the known sequence, can be 
readily determined. Such sequences may shorten the time necessary to train a 
nonlinear optical channel equalizer. 

A nonlinear channel estimator could also be built based on the look up table 
approach illustrated in Figure 4. The implementation of such a look up table version is 
a straight-forward adaptation of the channel characterization circuit illustrated in Figure 
4. 

Figure 15 is a block diagram of a nonlinear channel estimator combined with a 
Viterbi decoder. Instead of a decision feedback equalizer a Viterbi decoder may be 
used. Nonlinear channel estimator 1 501 would be used to compute the expected signal 
going to the branch metric computations 1503. The remainder of the Viterbi decoder 
system including the actual Viterbi decoder itself, block 1505, is similar to the linear 
case. The nonlinear channel estimator 1501 is creating a model of the received signal. 
If the nonlinear channel estimator 1501 has a memory comprising k bits, that is it 
remembers the last k inputs, then the Viterbi decoder 1505 would have 2< k1 > states. 
Therefore, if k=3 then the Viterbi decoder 1505 would be a 4 state Viterbi decoder. If 
the nonlinear channel estimator 1 501 has a memory of 3 bits then it can support eight 
different expected values of the received signal. Which would be implemented with a 
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1 look up table have eight locations. The expected values of the received signal are tised 
by the branch metric computations 1 503 to compute all the branch metrics in the Viterbi 
decoder. 

The Viterbi decoder illustrated in Figure 1 5 is the same as a standard type Viterbi 
5 decoder except for the fact that the channel estimator function is a nonlinear channel 
estimator with memory. The nonlinearity is expressed either in the form of a look up 
table or a Volterra Kernel. 

The Viterbi decoder includes a receiver for receiving a signal including linear and 
non linear components. A non linear channel estimator computes the expected values 
10 of the received signal. A branch metrics computer computes the branch metrics based 
on the expected values of the received signal. A standard Viterbi decoder accepts the 
computed branch metrics and decodes the received signal. The non linear channel 
estimator can be fabricated using a Volterra kernel estimator or a lookup table estimator 
as illustrated at 1401 as illustrated at 1433 in Figure 14A or a look up table 
15 implementation as illustrated at 1433 in Figure 14B. 
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WHAT IS CLAIMED IS: 

1. A method for modeling the behavior of a data channel the method 
comprising: 

determining a sequence of data input to the data channel; 
using at least part of the sequence of data input to the data channel as an index 
to a channel model value; 

sampling the data after it has passed through the channel to produce a sampled 

value; . 

comparing the channel model value with the sampled value; and 
adjusting the channel model value based on the results of the comparison 
between the channel model value and the sampled value. 

2. Amethod as in claim 1 wherein determining a sequence of data inputto the 
data channel comprises determining the last N bits input to the channel. 

3. A method as in claim 2, where N = 5. 

4. A method as in claim 1 wherein the sampling of the data after it has passed 
through the channel to produce a sampled value comprises producing a real number 
representing the sampled value. 

5. A method as in claim 1 , wherein adjusting the channel model value further 
comprises adjusting the channel model value according to an LMS (Least Means 
Squared) algorithm. 

6. A method as in claim 1 wherein adjusting the channel model value further 
comprises adjusting the channel model value until it converges. 

7. A method as in claim 6 further comprising converting the look up table into 
Volterra Kernels. 

8. A method as in claim 7 further comprising converting the look up table into 
Volterra Kernels using a Hadamard transform. 

9. A method as in claim 7 further comprising adjusting the Volterra Kernels 
based on the results of the comparison between the channel model value and the 
sampled value. 
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10. A method as in claim 9 further comprising eliminating the insignificant 
Volterra Kernels. 

11. A method for modeling the behavior of a data channel the method 
comprising: 

determining a sequence of data input to the data channel; 
determining a Volterra Series representation of the channel; 
accepting at least part of the sequence of data input to the data into the Volterra 
series representation of the channel to produce a channel model value; 

sampling the data after it has passed through the channel to produce a sampled 

value; 

comparing the channel model value with the sampled value; and 
adjusting the channel model value based on the results of the comparison 
between the channel model value with the sampled value. 

12. A method as in claim 11 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 

accepting the most recent value of the sequence of data input to the data channel 
into a first FIR (Finite Impulse Response) filter; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and a second most recent value of the sequence of data input to the 
data channel into a second FIR; and 

summing an output of the first FIR and output of the second FIR to form the 
channel model value. 

13. A method as in claim 11 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 

accepting the most recent value of the sequence of data input to the data channel 
into a first FIR filter; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and the second most recent value of the sequence of data input to the 
data channel into a second FIR; 

accepting a product of the most recent value of the sequence of data input to 
the data channel and a third most recent value of the sequence of data input to the data 
channel into a third FIR; and 

summing an output of the first FIR and output of the second FIR and output of the 
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third FIR to form the channel model value. 

14. A method as in claim 11 wherein determining a Volterra Series 
representation of the channel comprises: 

accepting a most recent value of the sequence of data input to the data channel; 

accepting the most recent value of the sequence of data input to the data channel 
into a first FIR filter; 

accepting a product of the most recent value of the sequence of data input to the 
data channel and the second most recent value of the sequence of data input to the 
data channel into a second FIR; 

accepting a product of the most recent value of the sequence of data input to the 
data channel and a third most recent value of the sequence of data input to the data 

channel into a third FIR; 

accepting a product, said product being the most recent value of the sequence 
of data input to the data channel and the two next most recent data input, into a fourth 
FIR; and 

summing an output of the first FIR and output of the second FIR and output of the 
third FIR and output of the fourth FIR to form the channel model value. 

15. A method as in claim 1 1 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

16. A method as in claim 1 2 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

17. A method as in claim 1 3 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

18. A method as in claim 1 4 wherein the difference between the channel model 
value and the output of the channel is used to update all the FIRs. 

19. A method as in 1 5 wherein an LMS algorithm is used to update all the FIRs. 

20. A method as in 16 wherein an LMS algorithm is used to update all the FIRs. 

21 . A method as in 17 wherein an LMS algorithm is used to update all the FIRs. 
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22. A method as in 18 wherein an LMS algorithm is used to update all the FIRs. 

23. A method for equalizing an optical signal, modulated with a digital signal, 
received over an optical channel, the method comprising: 

converting the optical signal into an electrical signal; 
summing the electrical signal with a correction signal; 
providing the summed signal to a detector; 
detecting the summed signal to produce decisions; 
providing the decisions to a nonlinear channel estimator; and 
estimating the correction signal in the nonlinear channel estimator. 

24. The method of clam 23 wherein estimating the correction signal in the 
nonlinear estimator further comprises: 

accepting the decisions; 

predicting the inter-symbol interference of the channel in a nonlinear channel 
estimator; and 

forming a correction signal from the predicted inter-symbol interference. 

25. The method of claim 24 wherein predicting the inter-symbol interference 
of the channel further comprises: 

providing the decisions to a plurality of Volterra Kernels; and 

summing the output of the plurality Volterra Kernels to form a correction signal. 

26. The method of claim 25 further comprising: 

comparing the predicted inter-symbol interference to inter-symbol interference in 
the electrical signal; and 

updating the Volterra Kernels based on the result. 

27. The method as in claim 26 wherein updating the Volterra Kernels 
comprises using a LMS (Least Means Squared) algorithm to update the Volterra 
Kernels. 

28. The method of claim 24 wherein predicting the inter-symbol interference 
of the optical channel further comprises: 

providing the data decisions as an address into a look up table; 
qutputting a value stored in the look up table as the predicted inter-symbol 
interference; 

comparing the predicted inter-symbol interference to the inter-symbol interference 
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in the electrical signal; and 

updating the value stored in the look up table based on the result. 

29. The method as in claim 28 wherein updating the value stored in the look 
up table comprises using a LMS (Least Means Squared) algorithm. 

30. An apparatus for equalizing a signal received over an optical channel, the 

apparatus comprising: 

an input that accepts an optical signal; 

a converter that converts the optical signal into an electrical signal; 
a summation unit that sums the electrical signal with a correction signal; 
a detector that detects the summed signal to produce decisions; and 
a nonlinear channel estimator that estimates the correction signal. 

31. The apparatus of claim 30 wherein the nonlinear estimator further 

comprises: 

an input that accepts the decisions; 

an estimating circuit that predicts the inter-symbol interference of the channel; 

and 

an output that provides a correction signal from the estimated inter-symbol 
interference. 

32. The apparatus of claim 31 wherein estimating circuit that estimates the 
- intersymbol interference of the channel further comprises: 

a plurality of Volterra kernels that accept the decisions and produce individual 
outputs; and 

a summation unit that sums the individual outputs of the plurality of Volterra 
Kernels to form a correction signal. 

33. The apparatus of claim 32 further comprising: 

a comparator for comparing the predicted inter-symbol interference to the inter- 
symbol interference in the electrical signal; and 

means for updating the Volterra kernels based on the result. 

34. The apparatus as in claim 33 wherein the comparator that compares the 
predicted inter-symbol interference to the inter-symbol interference in the electrical 
signal includes a LMS (Least Means Squared) algorithm that compares the predicted 
inter-symbo) interference to the inter-symbol interference in the electrical signal. 
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35. The apparatus of claim 31 wherein the estimating circuit that estimates the 
intersymbol interference further comprises: 

a look up table which accepts the data decisions as an address into the look up 

table; 

values of the predicted inter-symbol interference stored in the look up table; 

a subtractor that subtracts the predicted inter-symbol interference from the 
electrical signal to produce a signal substantially free from intersymbol interference; 

a second subtractor that subtracts the decision from the substantially inter-symbol 
interference free signal to produce an error; 

and 

means for updating the value stored in the look up table based on the error. 

36. The apparatus as in claim 35 wherein comparing the comparator that 
compares the estimated inter-symbol interference to the inter-symbol interference in the 
electrical signal comprises a LMS (Least Means Squared) algorithm that compares the 
predicted inter-symbol interference to the inter-symbol interference in the electrical 
signal. 

37. A method for decoding a signal received over an optical channel, the 
method comprising: 

receiving a signal including linear and non linear components; 
estimating, in a non linear channel estimator, the expected values of the received 
signal; 

computing the branch metrics based on the expected values of the received 
signal; 

providing the computed branch metrics to a Viterbi decoder; and 
Viterbi decoding the received signal using the branch metrics provided to the 
Viterbi decoder. 

38. The method of claim 37 wherein estimating, in a non linear channel 
estimator, the expected values of the received signal comprises: 

providing the value of the received signal to a Volterra kernel estimator; and 
computing the expected value sent based on the output of the Volterra kernel 
estimator. 

39. The method of claim 37 wherein estimating, in a non linear channel 
estimator, the expected values of the received signal comprises: 

providing the value of the received signal as an address to a look up table; and 
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40. An apparatus for decoding a signal received over an optical channel, the 
apparatus comprising: 

a receiver for receiving a signal including linear and non linear components; 
a non linear channel estimator that computes the expected values of the received 
signal; 

a branch metrics computer for computing the branch metrics based on the 
expected values of the received signal; 

a Viterbi decoder that accepts the computed branch metrics and 
Viterbi decodes the received signal. 

41 . The apparatus of claim 40 wherein the non linear channel estimator that 
computes the expected values of the received signal comprises a Volterra kernel 
estimator that computes the expected value sent based on the output of the Volterra 
kernel estimator. 

42. The apparatus of claim 40 wherein the non linear channel estimator that 
computes the expected values of the received signal comprises a look up table that 
uses the value of the received signal as an address to look up the stored value as the 
actual value transmitted . 

43. A method for detecting digital data modulated on an optical signal and 
received over an optical channel, the method comprising: 

converting the optical signal to an electrical signal; 
converting the electrical signal to a multibit digital representation; 
estimating distortion introduced in the optical signal by the optical channel; 
compensating the multibit digital representation for the distortion; and 
detecting the digital data from the compensated multibit digital representation. 

44. A method as in claim 43 wherein estimating distortion introduced in the 
optical signal by the optical channel comprises estimating in a Volterra Kernel estimator 
the distortion introduced in the optical channel. 

45. A method as in claim 43 wherein estimating distortion introduced in the 
optical signal by the optical channel comprises estimating in a lookup table estimator 
the distortion introduced in the optical channel. 
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46. An apparatus for detecting digital data modulated on an optical signal and 
received over an optical channel, the apparatus comprising: 

a converter for converting the optical signal to an electrical signal; 
an analog to digital converter that converts the electrical signal to a multibit digital 
representation; 

an estimator that estimates distortion introduced in the optical signal by the 
optical channel; 

a compensator that compensates the multibit digital representation for the 
distortion; and 

a detector that detects the digital data from the compensated multibit digital 
representation. 

47. A method as in claim 45 wherein estimator that estimates distortion 
introduced in the optical signal comprises a Volterra Kernel estimator. 

48. A method as in claim 45 wherein estimator that estimates distortion 
introduced in the optical signal comprises a lookup table estimator. 

49. An apparatus as in claim 46 wherein the estimator that estimates distortion 
introduced in the optical signal comprises a Volterra Kernel estimator. 

50. An apparatus as in claim 46 wherein estimator that estimates distortion 
introduced in the optical signal comprises a lookup table estimator. 
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